Census data is central in situating analysis, whether we want to make use of census data directly, mix it with our own data or just use it to calibrate external data we have.
In this workshop we’ll explore how to work with census data and use it in conjunction with our own data.
Census Data
Census data offers rich variables at high spatial resolution, but at coarse time intervals.
Richer data comes at a price: Data discovery and acquisition is more complex. Enter CensusMapper.
CensusMapper is a flexible census data mapping platform. Anyone can explore and map census data.
CensusMapper is also an API server to facilitate data acquisition for analysis, as a GUI data selection tool.
CanCensus Package
The cancensus R package interfaces with the CensusMapper API server. It can be queried for:
census geographies
census data
hierarchical metadata of census variables
some non-census data that comes on census geographies, e.g. T1FF taxfiler data
Setting up API Key
A slight complication, the cancensus package needs an API key. You can sign up for one on CensusMapper, and install it using the set_api_key function with the install=TRUE option so it’s always available and won’t expose your API key when sharing code.
This code is commented-out for now because you only need to run it once. When you get your API key, place it above and run that section of code without the # sign. It should look something like this:
This will install the API key as a system variable in your .Renviron so that it’s available in every R session and you won’t expose your API key when sharing code.
Setting up the Cache
This step will store any census data you grab locally, so that you don’t have to keep re-downloading it every time. It’s really helpful for speeding up code when you’re working with bigger tables, especially ones that you use often.
You’ll see a use_cache argument in most of the functions we use next. That determines whether or not the code uses the data from your local cache or not. It’s set to TRUE by default, so it uses the data already stored on your computer if it’s already there.
To force cancensus to refresh the data and re-download it from StatCan, you can specify use_cache = FALSE as a parameter for the functions we’ll learn about next.
Testing it out
cancensus provides three different functions for retrieving Census data:
get_census_data to retrieve Census data only as a flat data frame
Cancensus can access Statistics Canada Census data for Census years 1996, 2001, 2006, 2011, 2016, and 2021. You can run list_census_datasets to check what datasets are currently available for access through the CensusMapper API.
Census data is aggregated at multiple geographic levels. Census geographies at the national (C), provincial (PR), census metropolitan area (CMA), census agglomeration (CA), census division (CD), and census subdivision (CSD) are defined as named census regions.
Canadian Census geography can change in between Census periods. Cancensus provides a function, list_census_regions(dataset), to display all named census regions and their corresponding id for a given census dataset.
This is how we got to the function from above. To grab the same data for Edmonton, broken down into a smaller geographic level, we can slightly modify the level= argument.
Census data contains thousands of different geographic regions as well as thousands of unique variables. In addition to enabling programmatic and reproducible access to Census data, cancensus has a number of tools to help users find the data they are looking for.
You can run the following code to view all available Census variables for a given dataset:
list_census_vectors("CA21")
# A tibble: 7,709 × 7
vector type label units parent_vector aggregation details
<chr> <fct> <chr> <fct> <chr> <chr> <chr>
1 v_CA21_1 Total Population, 2021 Numb… <NA> Additive CA 202…
2 v_CA21_2 Total Population, 2016 Numb… <NA> Additive CA 202…
3 v_CA21_3 Total Population percenta… Numb… <NA> Average of… CA 202…
4 v_CA21_4 Total Total private dwell… Numb… <NA> Additive CA 202…
5 v_CA21_5 Total Private dwellings o… Numb… v_CA21_4 Additive CA 202…
6 v_CA21_6 Total Population density … Ratio <NA> Average of… CA 202…
7 v_CA21_7 Total Land area in square… Numb… <NA> Additive CA 202…
8 v_CA21_8 Total Total - Age Numb… <NA> Additive CA 202…
9 v_CA21_9 Male Total - Age Numb… <NA> Additive CA 202…
10 v_CA21_10 Female Total - Age Numb… <NA> Additive CA 202…
# ℹ 7,699 more rows
Working with Census Variables
For each variable (vector) in that Census dataset, this shows:
Vector: short variable code
Type: variables are provided as aggregates of female responses, male responses, or total (male+female) responses
Label: detailed variable name
Units: provides information about whether the variable represents a count integer, a ratio, a percentage, or a currency figure
Parent_vector: shows the immediate hierarchical parent category for that variable, where appropriate
Aggregation: indicates how the variable should be aggregated with others, whether it is additive or if it is an average of another variable
Description: a rough description of a variable based on its hierarchical structure. This is constructed by cancensus by recursively traversing the labels for every variable’s hierarchy, and facilitates searching for specific variables using key terms
Finding Variables to Work With
As you can tell, it’s pretty hard to find a dataset on your own just by browsing that list. Cancensus uses the find_census_vectors() function to help with that.
# A tibble: 3 × 4
vector type label details
<chr> <fct> <chr> <chr>
1 v_CA21_4812 Total Australia 25% Data; Citizenship and immigration; Total - P…
2 v_CA21_5223 Total Australian 25% Data; Visible minority and ethnic origin; To…
3 v_CA21_6483 Total Australia 25% Data; Education; Total - Location of study c…
# A tibble: 251 × 4
vector type label details
<chr> <fct> <chr> <chr>
1 v_CA21_4917 Total Total - Ethnic or cultural origin for the populati… 25% Da…
2 v_CA21_4920 Total Canadian 25% Da…
3 v_CA21_4923 Total English 25% Da…
4 v_CA21_4926 Total Irish 25% Da…
5 v_CA21_4929 Total Scottish 25% Da…
6 v_CA21_4932 Total French, n.o.s. 25% Da…
7 v_CA21_4935 Total German 25% Da…
8 v_CA21_4938 Total Chinese 25% Da…
9 v_CA21_4941 Total Italian 25% Da…
10 v_CA21_4944 Total Indian (India) 25% Da…
# ℹ 241 more rows
The “exact” search is very precise, but you can miss out on key tables if you don’t know exactly what you’re looking for.
Finding Variables to Work With
One other search option is “keyword”, which looks for the highest number of matches. It also has an interactive option that you can play around with:
find_census_vectors("Australian ethnic", dataset ="CA21", type ="total", query_type ="keyword", interactive =FALSE)
# A tibble: 1 × 4
vector type label details
<chr> <fct> <chr> <chr>
1 v_CA21_5223 Total Australian 25% Data; Visible minority and ethnic origin; To…
# A tibble: 1 × 8
region name level pop municipal_status CMA_UID CD_UID PR_UID
<chr> <chr> <chr> <int> <chr> <chr> <chr> <chr>
1 48835 Edmonton CMA 1418118 B <NA> <NA> 48
find_census_vectors("Low Income Measures",dataset="CA21",type="total",query_type="semantic") %>% knitr::kable()
vector
type
label
details
v_CA21_1025
Total
In low income based on the Low-income measure, after tax (LIM-AT)
Income; Low income status; LIM-AT; In low income based on the Low-income measure, after tax (LIM-AT)
v_CA21_1028
Total
0 to 17 years
Income; Low income status; LIM-AT; In low income based on the Low-income measure, after tax (LIM-AT); 0 to 17 years
v_CA21_1031
Total
0 to 5 years
Income; Low income status; LIM-AT; In low income based on the Low-income measure, after tax (LIM-AT); 0 to 17 years; 0 to 5 years
v_CA21_1034
Total
18 to 64 years
Income; Low income status; LIM-AT; In low income based on the Low-income measure, after tax (LIM-AT); 18 to 64 years
v_CA21_1037
Total
65 years and over
Income; Low income status; LIM-AT; In low income based on the Low-income measure, after tax (LIM-AT); 65 years and over
v_CA21_1040
Total
Prevalence of low income based on the Low-income measure, after tax (LIM-AT) (%)
Income; Low income status; LIM-AT; Prevalence of low income based on the Low-income measure, after tax (LIM-AT) (%)
v_CA21_1043
Total
0 to 17 years
Income; Low income status; LIM-AT; Prevalence of low income based on the Low-income measure, after tax (LIM-AT) (%); 0 to 17 years
v_CA21_1046
Total
0 to 5 years
Income; Low income status; LIM-AT; Prevalence of low income based on the Low-income measure, after tax (LIM-AT) (%); 0 to 17 years; 0 to 5 years
v_CA21_1049
Total
18 to 64 years
Income; Low income status; LIM-AT; Prevalence of low income based on the Low-income measure, after tax (LIM-AT) (%); 18 to 64 years
v_CA21_1052
Total
65 years and over
Income; Low income status; LIM-AT; Prevalence of low income based on the Low-income measure, after tax (LIM-AT) (%); 65 years and over
Mapping the Data
pv <-c(lico_at="v_CA21_1028")poverty_data <-get_census("CA21", regions=list(CMA="48835"), vectors=pv, geo_format="sf", level="CT")ggplot(poverty_data,aes(fill=lico_at)) +geom_sf(size=NA) +labs(title="Number of children in poverty - Edmonton",fill=NULL,caption="StatCan Census 2021")
Looking at Poverty (%)
find_census_vectors("Low Income Measures %",dataset="CA21",type="total",query_type="semantic") %>% knitr::kable()
vector
type
label
details
v_CA21_1025
Total
In low income based on the Low-income measure, after tax (LIM-AT)
Income; Low income status; LIM-AT; In low income based on the Low-income measure, after tax (LIM-AT)
v_CA21_1028
Total
0 to 17 years
Income; Low income status; LIM-AT; In low income based on the Low-income measure, after tax (LIM-AT); 0 to 17 years
v_CA21_1031
Total
0 to 5 years
Income; Low income status; LIM-AT; In low income based on the Low-income measure, after tax (LIM-AT); 0 to 17 years; 0 to 5 years
v_CA21_1034
Total
18 to 64 years
Income; Low income status; LIM-AT; In low income based on the Low-income measure, after tax (LIM-AT); 18 to 64 years
v_CA21_1037
Total
65 years and over
Income; Low income status; LIM-AT; In low income based on the Low-income measure, after tax (LIM-AT); 65 years and over
v_CA21_1040
Total
Prevalence of low income based on the Low-income measure, after tax (LIM-AT) (%)
Income; Low income status; LIM-AT; Prevalence of low income based on the Low-income measure, after tax (LIM-AT) (%)
v_CA21_1043
Total
0 to 17 years
Income; Low income status; LIM-AT; Prevalence of low income based on the Low-income measure, after tax (LIM-AT) (%); 0 to 17 years
v_CA21_1046
Total
0 to 5 years
Income; Low income status; LIM-AT; Prevalence of low income based on the Low-income measure, after tax (LIM-AT) (%); 0 to 17 years; 0 to 5 years
v_CA21_1049
Total
18 to 64 years
Income; Low income status; LIM-AT; Prevalence of low income based on the Low-income measure, after tax (LIM-AT) (%); 18 to 64 years
v_CA21_1052
Total
65 years and over
Income; Low income status; LIM-AT; Prevalence of low income based on the Low-income measure, after tax (LIM-AT) (%); 65 years and over
Mapping the Data
pv <-c(lico_at="v_CA21_1043")poverty_data <-get_census("CA21", regions=list(CMA="48835"), vectors=pv, geo_format="sf", level="CT")ggplot(poverty_data,aes(fill=lico_at/100)) +geom_sf(size=NA) +labs(title="Share of children in poverty - Edmonton",fill=NULL,caption="StatCan Census 2021")
# A tibble: 1 × 8
region name level pop municipal_status CMA_UID CD_UID PR_UID
<chr> <chr> <chr> <int> <chr> <chr> <chr> <chr>
1 59933 Vancouver CMA 2642825 B <NA> <NA> 59
poverty_data <-get_census("CA21", regions=list(CMA="59933"), vectors=pv, geo_format="sf", level="CT") ggplot(poverty_data,aes(fill=lico_at/100)) +geom_sf(size=NA) +labs(title="% of children in poverty - Vancouver",fill=NULL,caption="StatCan Census 2021")
Re-mapping with a Different Theme
If we want to add different themes to the graph, there are a lot of options to choose from:
poverty_data <-get_census("CA21", regions=list(CMA="59933"), vectors=pv, geo_format="sf", level="CT") ggplot(poverty_data,aes(fill=lico_at/100)) +geom_sf(size=NA) +labs(title="% of children in poverty - Vancouver",fill=NULL,caption="StatCan Census 2021") +scale_fill_viridis_c(option ="inferno",labels=scales::percent)
Mixing data sources
Mixing data sources is hard, especially when dealing with spatial data.
If spatial units match across datasets, it is easy to compare the other data we have. If spatial units don’t match, things get complicated. And annoying.
Simple example is LFS time series for CMAs. We don’t have one long time series but several partially overlapping shorter time series. The reason is that CMA geography changes over time, so we can’t directly compare data when geography (denominators) change.
Mixing Data when Spatial Units Match
The cansim package returns a geographic identifier GeoUID that matches census identifiers returned by cancensus. That makes matching data from those two data sources relatively easy.
We’ll try out an example here with an Income Distribution measure from StatCan:
Census geographies often change over time, which complicates comparisons using more than one year of data.
The best way to deal with this is a custom data request, but that takes time, costs money and is overkill for many applications. An immediate way to achieve almost the same result is using the tongfen package.
Tongfen ensures that while the spatial units change, they are still comparable. In other words, they’re derived from one another by a (generally short) series of split and join operations.
Tongfen & CensusMapper
With this example, we’ll take a look at how the number of children in Toronto has changed over time. We’ll use the CensusMapper API GUI to select the Census vectors that we need. We’ll get data on the children under 15, assembled from 5 year age groups for males and females for 2001 and 2021.
search_census_regions("Toronto","CA21")
# A tibble: 3 × 8
region name level pop municipal_status CMA_UID CD_UID PR_UID
<chr> <chr> <chr> <int> <chr> <chr> <chr> <chr>
1 35535 Toronto CMA 6202225 B <NA> <NA> 35
2 3520 Toronto CD 2794356 CDR <NA> <NA> 35
3 3520005 Toronto CSD 2794356 C 35535 3520 35
census_data <-get_census(dataset='CA21', regions=list(CSD="3520005"), vectors=c("v_CA21_11"), labels="detailed", geo_format=NA, level='CSD')variables <-c(children_2021="v_CA21_11",children1m_2001="v_CA01_7",children2m_2001="v_CA01_8",children3m_2001="v_CA01_9",children1f_2001="v_CA01_26",children2f_2001="v_CA01_27",children3f_2001="v_CA01_28")# "meta" vectors for Tongfen are the Census vectors that we need to compare over timemeta <-meta_for_ca_census_vectors(c(children_2021="v_CA21_11",children1m_2001="v_CA01_7",children2m_2001="v_CA01_8",children3m_2001="v_CA01_9",children1f_2001="v_CA01_26",children2f_2001="v_CA01_27",children3f_2001="v_CA01_28"))
Since the data from the 2001 Census is separated by 5 year age groups instead of 15, we’ll need to add them up to compare them with the 2021 Census Data.
This fancy code at the end gets rid of all the exrtra columns we had, since the original table was pretty clunky. The first part of the matches() function here is keeping only columns that start with “children_” and have exactly 4 numbers at the end (e.g. - children_2021). The second part keeps the columns that have “Population” in their name.
Analysis & Visualization
ggplot(plot_data, aes(fill=children_2021-children_2001)) +geom_sf() +scale_fill_gradient2(labels=scales::comma) +coord_sf(datum=NA) +labs(title="City of Toronto change in number of children under 15 between 2001 to 2021",fill="Number of\nchildren",caption="StatCan Census 2001, 2021")
Mapping it again Differently
ggplot(plot_data, aes(fill=children_2021/Population_CA21-children_2001/Population_CA01)) +geom_sf() +scale_fill_gradient2(labels=scales::percent) +coord_sf(datum=NA) +labs(title="City of Toronto change in share of children under 15 between 2001 to 2021",fill="Percentage\npoint\nchange",caption="StatCan Census 2001, 2021")
TongFen with T1FF data - Advanced Example
T1FF taxfiler data is a rich source of demographic information, now available for census tracts. Variable naming is consistent across years, making it possible to programmatically assemble data.
# A tibble: 8 × 4
vector type label details
<chr> <fct> <chr> <chr>
1 v_TX2004_607 Total All families (CF + LP) - # Tax data…
2 v_TX2004_608 Total Median total income Tax data…
3 v_TX2004_609 Total Person Median Income Tax data…
4 v_TX2004_610 Total # persons Tax data…
5 v_TX2004_622 Total All families (CF + LP) with employment income - # Tax data…
6 v_TX2004_623 Total Median employment income Tax data…
7 v_TX2004_632 Total All families (CF + LP) - # Tax data…
8 v_TX2004_633 Total Median Amount Tax data…
Advanced Example (continued)
find_census_vectors("# of families in low income",dataset="TX2004",query_type="semantic")
# A tibble: 21 × 4
vector type label details
<chr> <fct> <chr> <chr>
1 v_TX2004_786 Total # of families in Low Income - Total Tax data 2004; Table …
2 v_TX2004_782 Total With 0 children Tax data 2004; Table …
3 v_TX2004_783 Total With 1 child Tax data 2004; Table …
4 v_TX2004_784 Total With 2 children Tax data 2004; Table …
5 v_TX2004_785 Total With 3+ children Tax data 2004; Table …
6 v_TX2004_796 Total # of families in Low Income - Total Tax data 2004; Table …
7 v_TX2004_792 Total With 0 children Tax data 2004; Table …
8 v_TX2004_793 Total With 1 child Tax data 2004; Table …
9 v_TX2004_794 Total With 2 children Tax data 2004; Table …
10 v_TX2004_795 Total With 3+ children Tax data 2004; Table …
# ℹ 11 more rows
low_income_pivoted <- low_income %>%pivot_longer(starts_with("20"))low_income_pivoted %>% sf::st_sf() %>%ggplot(aes(fill=value)) +facet_wrap("name") +geom_sf(size=0.1) +scale_fill_gradient2(labels=scales::percent) +coord_sf(datum=NA,xlim=c(-123.25,-122.8),ylim=c(49.1,49.35)) +labs(title="Change in share of families in low income 2004-2018", fill=NULL,caption="T1FF F-20 family file")
Example on your Own
Now that we’ve gone through a lot of examples, we’ll have you all try out one on your own. In the meantime, I’ll show you through the steps again (if you need it) and answer any questions you have. Here’s the goal:
Find one Census Year & Vector that you want to analyze. Either with the CensusMapper in the browser, or with the find_census_vectors() function.
Pick a geography you’ll want to map. You can again use CensusMapper to grab the GeoID in the browser, or you can use the list_census_regions() function here in R.
Map the data. Choose the level of geographic detail you want with the level="CT/CMA/etc." code.
Good luck!
Extra Example of my Own:
pv <-c(households="v_CA21_4237")census_data <-get_census("CA21", regions=list(CMA="59933"), vectors=pv, geo_format="sf", level="CT")ggplot(census_data,aes(fill=households)) +geom_sf(size=NA) +labs(title="# of Households in Vancouver",fill=NULL,caption="StatCan Census 2021")+scale_fill_viridis_c()
